DOCUMENT RESUME 



ED 470 188 



IR 021 630 



AUTHOR 



TITLE 



PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Foley, Shawn; Remley, Christine; Morales, D. Roberto; Grega, 
Lawrence M.; Lantz, Nathaniel; Haughton, Noela A.; Shalyefu- 
Shimhopileni, R-Kavena 

Designing and Developing Rubrics as an Instrument To Assess 
ISTE*s (International Society for Technology in Education) 
National Educational Technology Standards for Teachers (NETS- 
T) . 

2001 - 11-00 

8p.; In: Annual Proceedings of Selected Research and 
Development [and] Practice Papers Presented at the National 
Convention of the Association for Educational Communications 
and Technology (24th, Atlanta, GA, November 8-12, 2001) . 
Volumes 1-2; see IR 021 504. Charts contain gray background. 
Reports - Descriptive (141) — Speeches/Meeting Papers (150) 
EDRS Price MFOl/PCOl Plus Postage. 

^Computer Literacy; ^Computer Uses in Education; ^Educational 
Technology; Higher Education; Information Skills; 
Instructional Design; *Preservice Teacher Education; 
^Standards 

* International Society for Technology in Education 



ABSTRACT 



Last year, the International Society for Technology in 
Education (ISTE) worked to develop and improve teacher skills in the area of 
educational technology by creating technology performance profiles for 
teacher preparation. These standards were given to a group of Pennsylvania 
State University instructional systems graduate students who developed 
rubrics to assess the four categories of standards: general preparation, 
professional preparation, student teaching/internship, and first-year 
teaching. The process and methods used in the design, development and 
creation of these rubrics is presented and discussed. (Author/AEF) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



IR021630 ED 470 188 






Designing and Developing Rubrics As An Instrument to Assess ISTE’s (International 
Society for Technology in Education) National Educational Technology Standards for 



Teachers (NETS-T) 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

p ■ 



Shawn Foley 
Christine Remley 
D, Roberto Morales 
MAJ Lawrence M, Grega 
Nathaniel Lantz 



^ U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
A CENTER (ERIC) 

M This document has been reproduced as 
7 I received from the person or organization 
/ originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

1 



Noela A. Haughton 
R-Kavena Shalyefu-Shimhopileni 
The Pennsylvania State University 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Abstract 

Last year, the International Society for Technology in Education (ISTE) worked to develop and improve 
teacher skills in the area of educational technology by creating technology performance profiles for 
teacher preparation. These standards were given to a group of PSU instructional systems graduate 
students who developed rubrics to assess the four categories of standards. The process and methods used 
in the design, development, and creation of these rubrics will be presented and discussed, and the rubrics 
will be available for use by the participants. 

Introduction 

Assessment has always been an important part of both the education and training fields. Standards are also 
valuable way of defining expectations, but when it comes to assessing whether standards have been achieved the 
plot thickens. Issues that arise concerning the reliability and validity of measurement and assessment will always be 
present. Assessment instruments are not always easy to design or develop. Some assessment instruments such as 
rubrics can be of some help. Rubrics are designed to “focus on measuring a stated objective (performance, behavior, 
or quality), use a range to rate performance, and contain specific performance characteristics arranged in levels 
indicating the degree to which a standard has been met.” (Pickett) A rubric may be applied in many situations and 
score criteria that is, “summative, by providing information about a student's knowledge, formative, by providing 
information about a student's strengths and weaknesses, evaluative, by providing ways to create instruction that 
better fits each student's needs, and educative, by providing students with an understanding of how they learn.” 

(Luft, 1997) Rubrics evaluate and measure a learning situation using a holistic or analytical approach that ideally 
results in the creation of an easy to use matrix of learning targets and the level at which these targets are achieved. 
The number of categories and levels of mastery is always up to the creator(s) of the rubric. 

This presentation discussed the use of rubrics as an instrument to assess standards. The National 
Educational Technology Standards (NETS) project writing team has created and defined technology performance 
profiles for teacher preparation across four categories in hopes to use these profiles in the development and 
improvement of teacher skills and competencies in the area of educational technology 

(http://cnets.iste.org/pdf/nets_brochure.pdf)- This project, initiated by the International Society for Technology in 
Education (ISTE), focused on the development of national standards for various uses of technologies that are 
associated with school improvement in the United States. A group of graduate students at Penn State University, 
taking a class based on analyzing outcomes and learners, given these standards and charged with developing 
assessment instruments based on the ISTE standards. This presentation explored the design and development of 
those assessment instruments. 

The presentation began with a brief introduction and statement of objectives. We then explored the work 
done by the ISTE NETS writing team with the intent to engage the audience in the specific areas addressed by these 
standards. We then explored the four categories of standards, general preparation, professional education, student 
teaching/intemship, and first-year teaching and the suggestions for evaluation provided by a class well versed in 
analyzing learners and outcomes. We looked at separate rubrics the class constructed and examined how they relate 
to the original standards. We discussed the design and development of these rubrics and the processes, obstacles, 
and methodologies in which these assessment instruments were created. The presentation ended with an open forum 
and a brief question and answer period between the audience and the presenters. 
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The format of this presentation recommends a refined approach, used for creating and implementing 
rubrics, as an appropriate instrument of assessment, specifically when evaluating performance based on the ISTE’s 
National Educational Technology Standards. The audience was expected to offer constructive criticism and 
suggested ways to improve upon the processes and methods used. 

The NETS-T Project 

• The National Educational Technology Standards for Teachers (NETS-T) is an International 
Society for Technology in Education (ISTE) initiative, which seeks to prepare today’s teachers -in- 
training to provide technology-supported learning opportunities and interventions for K-12 
classrooms. 

• This initiative is funded by a United States Department of Education’s Preparing Tomorrow’s 
Teachers to use Technology (PT^) Grant, a catalyst program designed to train technology literate 
teachers. 

• Accordingly, NETS-T defines technology knowledge and skills, as well as attitudes new teachers 
should possess as they proceed through their teacher preparation programs. 

• Each performance standard outlined in each of the four profiles is designed to assess ISTE’s 
National Education Technology Standards (NETS) and their corresponding Performance 
Indicators for teachers. 

• To facilitate this process, ISTE developed four profiles of technology literate teachers, each of 
which details the required technology knowledge and skills at the following stages of preparation: 

o Completion of the general preparation 
o Prior to intemship/student teaching experience 
o After completion of internship/student teaching 
o Upon completion of first year 




^Additional details for each profile are available at http://cnets.iste.org/pdf/nets brochure. pdf and at the 
ISTE website http://www.iste.org . 

Why We Did It 

• Kyle Peck, Ph.D., Professor of Education in Instructional Systems, assigned the assessment of the 
NETS-T, in partial fulfillment of a required graduate course in assessments. 
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• He felt that this project would both provide a real-life opportunity as well as be of practical value 
to ISTE’s efforts, which, to that date, had not created a comprehensive assessment strategy for this 
initiative. 

How It Was Done 

• For the most part, students were able to choose the Profile that they wished to develop assessments 
for. 

• We were given the freedom to organize the performance standards such that it was deemed 
manageable. 

• This meant combining, re -writing, and even removing performance standards that did not fit in 
with the respective graduate students assessment plan. 

• Accordingly, it is safe to say that no two-assessment products within each profile were the same. 



Examples 

Profile #2: Professional Preparation 
Description 

Learning Target III: Teaching, Learning and Curriculum (ISTE NETS Performance Indicators: 
I, III, IV, V) 

Performance Objective; The teacher will be able to design lessons and peer teach with technology that meets the 
content standards and reflects the best practices in teaching and learning. 

Performance Standard: Having observed teaching practices in various schools, video demonstrations and other peer 
teaching, the teacher will be able to design and teach lessons that reflect the best practices in teaching and learning 
using technology 

Assessment Method: Lesson Plans and Teaching Practice 

Performance Task: Design and teach technology enriched learning activities that maximize student leaning with 
diverse needs. 

Learning Target 1 

Elements of Assessment: Knowledge of Technology Concepts, Tools and their Uses. 

This is a ‘to do activity’ for your professional development. 

There are three sections numbered A, B, C. Please follow the instructions carefully. 

A. Technology Concepts. 

i) Assessing your conceptualization of technology-related concepts. 

Directions: Work through this activity and construct your own personal concepts about technology. Circle T or F to 
indicate what is True or False, and write an explanation if required. 

1. The terms ‘medium for instruction’ and ‘instructional technology’ are synonyms. 

TorF 

2. There is a difference between interactive technologies and presentation technologies. 

TorF 

3. Printed textbooks are technology tools for delivery. 

TorF 
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4. There is no significant difference in learner achievement between those who are taught with the 
Internet and those taught with traditional methods. 

T or F 

Explain your answer. 



ii) Assessing Personal Attitudes Towards Technology 

Directions: Place an “x” on the continuum to represent your feelings towards the statements below. 

1. Technology should be integrated into teaching and learning from the cradle to the grave. 

Agree Uncertain Disagree 

2. New technologies for use in the classroom are more important than traditional basic supplies. 

Agree Uncertain Disagree 

I ^ I 

3 . Computer-mediated teaching will soon make teachers redundant. 

Agree Uncertain Disagree 

I ^ I 

4. Today’s learners, who are more literate than their own teachers, threaten me. 

Agree Uncertain Disagree 

5. I am not confident with my skills to try new technologies. 

Agree Uncertain Disagree 

L_ ^ ^ I 

6. Today’s learners, who are very computer savvy, are less creative than learners who are less exposed to 
computers. 

Agree Uncertain Disagree 
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C. Technological Tools 



Assess your knowledge of Technological Tools. 

In the second column of the table below, place a checkmark next to the technological tools that you are 
comfortable using and make a cross by those you would like to be able to use. In the third column, give your 
comments on how you plan to improve your skills to use those tools. 



Medium 


Technology tools for Delivery 


Comments for Improvement 


Face-to-Face contact 


Presentation Aids: 
Black or white board 
Overhead projector 
Data projector 
Slide projector 




Text (including graphics) 


Printed textbook 

Web-based textbook 

Facsimile 
Computers 

(Including a range of applications such as e- 
mail, electronic databases. World Wide Web 
hypertext documents, FTP or ASCII documents, 
CD-ROM) 




Audio 


Audiocassette 
Radio broadcast 
Telephone 

Computers fwith related applications'! 




Video 


Television Broadcast 

(Terrestrial, satellite or cable and including 
narrowcast educational television'! 
Videocassettes 
Video disc 

Videoconferencing 




Integrated Multimedia 


Computer-mediated learning environments 
(This could be stand alone or networked and 
with numerous possible ways to access all 
media through a single technology) 





Knowledge of Computer Operations 

Directions: In the right-hand column, circle the letter(s) that indicate your knowledge about computer uses. 
K = I Know DK = I Don’t Know 



Graphic User 
Interface 


Performance Task 






Use of toolbar, 
menus, and dialog 
boxes 


Locate and use features on a toolbar. 
Locate and select items on menus. 
Locate and respond to dialog boxes. 


K 


DK 


Menu Shortcuts 


Identify and use menu shortcuts. 


K 


DK 


Selecting 


Select tools and other features within a GUI. 


K 


DK 


Click, Double 
CHck/Rifiht Click, 


Demonstrate clicking, double-clicking, right clicking (on PC), command 
clicking (On MAC), and dragging. 


K 


DK 
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Drag 






Window 


Identify and use the functions of a window. 


K 


DK 


Scrolling 


Scroll up and down a window. 


K 


DK 


Resizing 


Resize a window. 


K 


DK 


Closing 


Close a window. 


K 


DK 


Application 

Software 


Performance Task 




Launching 


Launch an application. 


K 


DK 


Creating files 


Create a file. 


K 


DK 


Saving 


Save a file. 


K 


DK 


Common features 


Identify the features that most applications have in common. 


K 


DK 


Basic four 

productivity 

applications 


Performance Task 




Word processing 


Demonstrate the use of a word processing application. 


K 


DK 


Spreadsheet 


Demonstrate the use of a spreadsheet application. 


K 


DK 


Database 


Demonstrate the use of a database application. 


K 


DK 


Graphics 


Use a graphic application. 


K 


DK 


File 

Management 


Performance Task 




Saving a file on a 
hard drive 


Save a file on a hard drive. 


K 


DK 


Saving a copy of 
same file at 
different location 


Save a copy of same file at different location. 


K 


DK 


Creating file 
folders 


Create a new file folder. 
Save/Move files to a folder. 


K 


DK 


Finding a 
directory from a 
pathname 


Find a directory from a pathname. 


K 


DK 


Naming 

conventions 


Recognize and use file -naming conventions (as they may appear on a MAC 
and a PC). 


K 


DK 




Explain the importance of using the correct naming conventions. 






File Recovery 


Identify and demonstrate the methods available to recover files and lost data. 


K 


DK 



Teaching, Learning and Curriculum 

Element of Assessment: Lesson Plan and Teaching Practice Using Technology 
Performance Task: Design and Teach Technology Enriched Learning Activities That Maximize 
Student Learning With Diverse Needs. 

Name of Rubric: Checklist for Lesson Observation and Assessment Using Interactive and 
Presentation Technologies 



Last Name: Initials: 

Student Id: 

Program of Study: 

Subject: Topic: 



Grade: Date: Time: 

Signature of Evaluator/Observer: Marks: 5x2= 10 (Each element 

carries 2 marks) 



A, LESSON PLAN PREPARATION 



DESCRIPTORS 



COMMENTS 
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Appropriate technological tools were chosen. 

Technology relevant to the topic and logically integrated in the lesson. 



B. LESSON PRESENTATION 



DESCRIPTORS 


COMMENTS 


Effective and skillful use of technological tools 

Technology chosen is purposeful and gained immediate attention from the 
class. 

Achieved appropriate content of the lesson. 





C. HANDLING OF TECHNOLOGICAL EQUIPMENT 



DESCRIPTORS 


COMMENTS 


Knew all the operations of the technological tools 





D. SENSITIVITY TO DIVERSE NEEDS 



DESCRIPTORS 


COMMENTS 


Technology used was sensitive to the special needs of individual identities. 
Technological tool used communicated to different sectors of the class. 





E. GENERAL IMPRESSION OF THE STUDENT TEACHER 



DESCRIPTORS 


COMMENTS 


Used a variety of technological tools and looked confident, enthusiastic and 
enjoys teaching with technology. 





AECT Presentation Conclusion 

The presentations were based on the International Society for Technology in Education (ISTE) National 
Educational Technology Standards (NETS) for Teachers. 

The class included students from varying professional backgrounds. We believe that all students, 
regardless of professional and academic background, found the assessment of these standards important to the 
success of their implementation. In evaluating these standards, most students found that creating an assessment 
process was difficult because it involved a great deal of compromise as well as huge amounts of time to complete 
the assessment of a standard. 

Despite the difficulties it was an extremely rewarding process because we were given experience of the 
difficulties and challenges inherent in any educational assessment. 
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